Hi all,
Sorry, I only just reviewed the Survey Monkey so far, so please disregard my suggestions at this point if you like. I think the open-ended q's are right on, but still wonder about our rating system (see below for my thoughts)...JJ
Nielsen also reminds us: "Survey bloat is a natural consequence of having a diverse group of marketing managers, all of whom want customer feedback on their special issues. Please resist the temptation to collect all the information that anybody could ever want. You will end up with no information (or misleading information) instead.
The simplest solution to survey bloat is to ask fewer questions. Ask questions that address only your core needs and skip the subtleties. Surveys are not great at gauging minor differences anyway — you need direct observation for that."
URL: http://www.useit.com/alertbox/20040202.html
Hi all,
Thanks for so much brainstorming already! I wonder if we need to be quite so detailed about every little thing in our design? I would suggest that we focus on the main, pertinent issues ( such as, preference of tutorial format, use and clarity of voice over, written instructions, clarity of directions, meanings of icons match the associated text, and the like) As Nielsen states:
"Beware the Query Effect
Whenever you do ask users for their opinions, watch out for the query effect: People can make up an opinion about anything, and they'll do so if asked. You can thus get users to comment at great length about something that doesn't matter, and which they wouldn't have given a second thought to if left to their own devices.
It's dangerous to make big design changes because "users didn't like this" or "users asked for that." If you ask leading questions or press respondents for answers, they might make up opinions that don't reflect their real preferences in the slightest.
For example, if you quiz people about your visual design, you'll inevitably get comments about the colors, even if they're not particularly important to the users. On the other hand, if you hear people mention the colors (unprompted) while they're using the site, then there's probably something to consider. (Say, a comment like "Wow, this neon blue really hurts the eyes," or a more positive statement like "The ultramarine is nice and calming.") at URL: http://www.useit.com/alertbox/interviews.html
I was wondering about the way statements and choice is expressed in the questionnaire format on page 312 of Preece et al. See what you think...can this style be replicated in Survey Monkey? So for, example, Strongly agree, Agree, Neutral, disagree, Strongly Disagree:
The written instructions in the tutorial are clear and easy to follow
The recorded instructions in the tutorial are clear and easy to follow
The tutorial clearly outlines how to add pictures when creating a new story
The visual layout of the tutorial is informative and helpful
Each icon has a clear and helpful meaning
The animated tutorial is a reasonable length
The written tutorial is a reasonable length
etc. if needed? if you like?
There should then be a section for more open-ended responses: Such as for comparison of the tutorials for preferences, effectiveness, and efficiency (for learning)? Or, add any comments or recommendations for changes to overall design and the like? Here is where Robin's question would be inserted:
You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why?
JJ
Hi everyone,
Thought I'd get a jump on the questionnaire. We can use this page to record possible questions. As noted in the Production Paper, we were going to ask open-ended questions regarding the effectiveness of the tutorial, ease of use, memorability?, learnability?, pacing?, comparing the two formats, as well as questions about icon clarity, voice over clarity, instructional vocabulary ... anything else? anything less?
I started a survey in Survey Monkey that we can use for the survey - if people like that idea - it would make collating/translating information easier, I think.
Okay, I'll get it started:
You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why?
Okay, I must be avoiding something - I went ahead and created some questions in Survey Monkey - I'll send a link/or link it here - depending on which I can manage! Take a look - it's a starting point - we can add/change, edit, delete and so on.
Thanks!
Beginning survey link:
http://www.surveymonkey.com/s/K2PD7ZF Hello all, I like the questions in the Survey Monkey, I was wondering if some pictures or graphics could also be shown for the users to recollect which tutorial we are referring and especially when they have to remember the aesthtics and ease of use?? Just a thought.. Though I'm not quite sure if survey Monkey would allow this feature? since we are so involved in the project, answers would be very natural to us, however I was thinking from the point of view of a novice user?
1. Icons- a) Did you like the icons in the tutorial?
b) Were the icons clear?
c) Is there any icon you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.
2. Colour- a)Did you like the use of colour in the tutorial?
b) Was the colour too bright ?
c) Was there any colour you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.
3. Voice- A ) Did you like the use of narration in the tutorial?
B) Was the voice clear and instructions easy to follow ?
c) Was there any aspect in the voice narration you did not like or was annoying ?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of voice is excellent.
4. Animation- a) Did you like the use of static animation in the tutorial? (the one without the voice)
b) Was the animation clear ?
c) Was there any part in the animation you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of animation is excellent.
5. (Semantic differential Scale)
Instructions: for each pair of adjectives, place a cross at the point between them that reflects the extent to which you believe the adjectives describe the Static tutorial, (the one without voice). You should place only one cross between the marks on each line.
Poor l………..l………….l………..l………..l………….l…………..l………….l Well designed
6.
Instructions: for each pair of adjectives, place a cross at the point between them that reflects the extent to which you believe the adjectives describe the Dynamic tutorial, (the one with voice, text, graphics and colour). You should place only one cross between the marks on each line.
Sorry, I only just reviewed the Survey Monkey so far, so please disregard my suggestions at this point if you like. I think the open-ended q's are right on, but still wonder about our rating system (see below for my thoughts)...JJ
Nielsen also reminds us:
"Survey bloat is a natural consequence of having a diverse group of marketing managers, all of whom want customer feedback on their special issues. Please resist the temptation to collect all the information that anybody could ever want. You will end up with no information (or misleading information) instead.
The simplest solution to survey bloat is to ask fewer questions. Ask questions that address only your core needs and skip the subtleties. Surveys are not great at gauging minor differences anyway — you need direct observation for that."
URL: http://www.useit.com/alertbox/20040202.html
Hi all,
Thanks for so much brainstorming already! I wonder if we need to be quite so detailed about every little thing in our design? I would suggest that we focus on the main, pertinent issues ( such as, preference of tutorial format, use and clarity of voice over, written instructions, clarity of directions, meanings of icons match the associated text, and the like) As Nielsen states:
"Beware the Query Effect
Whenever you do ask users for their opinions, watch out for the query effect: People can make up an opinion about anything, and they'll do so if asked. You can thus get users to comment at great length about something that doesn't matter, and which they wouldn't have given a second thought to if left to their own devices.It's dangerous to make big design changes because "users didn't like this" or "users asked for that." If you ask leading questions or press respondents for answers, they might make up opinions that don't reflect their real preferences in the slightest.
For example, if you quiz people about your visual design, you'll inevitably get comments about the colors, even if they're not particularly important to the users. On the other hand, if you hear people mention the colors (unprompted) while they're using the site, then there's probably something to consider. (Say, a comment like "Wow, this neon blue really hurts the eyes," or a more positive statement like "The ultramarine is nice and calming.") at URL: http://www.useit.com/alertbox/interviews.html
I was wondering about the way statements and choice is expressed in the questionnaire format on page 312 of Preece et al. See what you think...can this style be replicated in Survey Monkey? So for, example,
Strongly agree, Agree, Neutral, disagree, Strongly Disagree:
There should then be a section for more open-ended responses: Such as for comparison of the tutorials for preferences, effectiveness, and efficiency (for learning)? Or, add any comments or recommendations for changes to overall design and the like? Here is where Robin's question would be inserted:
You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why?
JJ
Hi everyone,
Thought I'd get a jump on the questionnaire. We can use this page to record possible questions. As noted in the Production Paper, we were going to ask open-ended questions regarding the effectiveness of the tutorial, ease of use, memorability?, learnability?, pacing?, comparing the two formats, as well as questions about icon clarity, voice over clarity, instructional vocabulary ... anything else? anything less?
I started a survey in Survey Monkey that we can use for the survey - if people like that idea - it would make collating/translating information easier, I think.
Okay, I'll get it started:
You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why?
Okay, I must be avoiding something - I went ahead and created some questions in Survey Monkey - I'll send a link/or link it here - depending on which I can manage! Take a look - it's a starting point - we can add/change, edit, delete and so on.
Thanks!
Beginning survey link:
http://www.surveymonkey.com/s/K2PD7ZF
Hello all,
I like the questions in the Survey Monkey, I was wondering if some pictures or graphics could also be shown for the users to recollect which tutorial we are referring and especially when they have to remember the aesthtics and ease of use?? Just a thought.. Though I'm not quite sure if survey Monkey would allow this feature?
since we are so involved in the project, answers would be very natural to us, however I was thinking from the point of view of a novice user?
1. Icons-
a) Did you like the icons in the tutorial?
b) Were the icons clear?
c) Is there any icon you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.
2. Colour-
a)Did you like the use of colour in the tutorial?
b) Was the colour too bright ?
c) Was there any colour you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.
3. Voice-
A ) Did you like the use of narration in the tutorial?
B) Was the voice clear and instructions easy to follow ?
c) Was there any aspect in the voice narration you did not like or was annoying ?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of voice is excellent.
4. Animation-
a) Did you like the use of static animation in the tutorial? (the one without the voice)
b) Was the animation clear ?
c) Was there any part in the animation you did not like?
d) Any suggestions for improvement?
e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of animation is excellent.
5. (Semantic differential Scale)
Attractive l………..l………….l………..l………..l………….l…………..l………….l Ugly
Clear l………..l………….l………..l………..l………….l…………..l………….l Confusing
Dull l………..l………….l………..l………..l………….l…………..l………….l Colourful
Exciting l………..l………….l………..l………..l………….l…………..l………….l Boring
Annoying l………..l………….l………..l………..l………….l…………..l………….l Pleasing
Helpful l………..l………….l………..l………..l………….l…………..l………….l Unhelpful
Poor l………..l………….l………..l………..l………….l…………..l………….l Well designed
6.
Attractive l………..l………….l………..l………..l………….l…………..l………….l Ugly
Clear l………..l………….l………..l………..l………….l…………..l………….l Confusing
Dull l………..l………….l………..l………..l………….l…………..l………….l Colourful
Exciting l………..l………….l………..l………..l………….l…………..l………….l Boring
Annoying l………..l………….l………..l………..l………….l…………..l………….l Pleasing
Helpful l………..l………….l………..l………..l………….l…………..l………….l Unhelpful
Poor l………..l………….l………..l………..l………….l…………..l………….l Well designed
· Ask them to perform the task from the beginning to the end by giving them instructions.( something similar we had done before)
· Roleplying pg. 330
.